AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	1.54749	1.88
	2_DecisionTree	Decision Tree	logloss	1.00623	19.38
the best	3_Default_Xgboost	Xgboost	logloss	0.817724	62.19
	4_Default_NeuralNetwork	Neural Network	logloss	0.94393	9.13
	5_Default_RandomForest	Random Forest	logloss	0.947229	35.05
	Ensemble	Ensemble	logloss	0.817724	0.64

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_Xgboost	3

Metric details

	0 to 17	18 to 29	30 to 49	50 to 69	70 or Older	accuracy	macro avg	weighted avg	logloss
precision	0.890017	0.506623	0.49328	0.517715	0.734665	0.631389	0.62846	0.627035	0.817724
recall	0.799392	0.334061	0.428238	0.582114	0.844955	0.631389	0.597752	0.631389	0.817724
f1-score	0.842274	0.402632	0.458463	0.548029	0.78596	0.631389	0.607472	0.624985	0.817724
support	658	458	857	1230	1219	0.631389	4422	4422	0.817724

Confusion matrix

	Predicted as 0 to 17	Predicted as 18 to 29	Predicted as 30 to 49	Predicted as 50 to 69	Predicted as 70 or Older
Labeled as 0 to 17	526	12	50	68	2
Labeled as 18 to 29	20	153	184	96	5
Labeled as 30 to 49	26	115	367	320	29
Labeled as 50 to 69	18	21	139	716	336
Labeled as 70 or Older	1	1	4	183	1030

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of 4_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• num_class: 5
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

8.1 seconds

Metric details

	0 to 17	18 to 29	30 to 49	50 to 69	70 or Older	accuracy	macro avg	weighted avg	logloss
precision	0.873857	0.452308	0.369674	0.497189	0.696085	0.585256	0.577823	0.578705	0.94393
recall	0.726444	0.320961	0.344224	0.503252	0.860541	0.585256	0.551084	0.585256	0.94393
f1-score	0.793361	0.375479	0.356495	0.500202	0.769626	0.585256	0.559033	0.577327	0.94393
support	658	458	857	1230	1219	0.585256	4422	4422	0.94393

Confusion matrix

	Predicted as 0 to 17	Predicted as 18 to 29	Predicted as 30 to 49	Predicted as 50 to 69	Predicted as 70 or Older
Labeled as 0 to 17	478	5	109	62	4
Labeled as 18 to 29	16	147	187	101	7
Labeled as 30 to 49	22	173	295	308	59
Labeled as 50 to 69	29	0	194	619	388
Labeled as 70 or Older	2	0	13	155	1049

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• num_class: 5
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

1.2 seconds

Metric details

	0 to 17	18 to 29	30 to 49	50 to 69	70 or Older	accuracy	macro avg	weighted avg	logloss
precision	0	0	0	0.278155	0	0.278155	0.0556309	0.07737	1.54749
recall	0	0	0	1	0	0.278155	0.2	0.278155	1.54749
f1-score	0	0	0	0.435244	0	0.278155	0.0870488	0.121065	1.54749
support	658	458	857	1230	1219	0.278155	4422	4422	1.54749

Confusion matrix

	Predicted as 0 to 17	Predicted as 18 to 29	Predicted as 30 to 49	Predicted as 50 to 69	Predicted as 70 or Older
Labeled as 0 to 17	0	0	0	658	0
Labeled as 18 to 29	0	0	0	458	0
Labeled as 30 to 49	0	0	0	857	0
Labeled as 50 to 69	0	0	0	1230	0
Labeled as 70 or Older	0	0	0	1219	0

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of 3_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: multi:softprob
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: mlogloss
• num_class: 5
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

60.9 seconds

Metric details

	0 to 17	18 to 29	30 to 49	50 to 69	70 or Older	accuracy	macro avg	weighted avg	logloss
precision	0.890017	0.506623	0.49328	0.517715	0.734665	0.631389	0.62846	0.627035	0.817724
recall	0.799392	0.334061	0.428238	0.582114	0.844955	0.631389	0.597752	0.631389	0.817724
f1-score	0.842274	0.402632	0.458463	0.548029	0.78596	0.631389	0.607472	0.624985	0.817724
support	658	458	857	1230	1219	0.631389	4422	4422	0.817724

Confusion matrix

	Predicted as 0 to 17	Predicted as 18 to 29	Predicted as 30 to 49	Predicted as 50 to 69	Predicted as 70 or Older
Labeled as 0 to 17	526	12	50	68	2
Labeled as 18 to 29	20	153	184	96	5
Labeled as 30 to 49	26	115	367	320	29
Labeled as 50 to 69	18	21	139	716	336
Labeled as 70 or Older	1	1	4	183	1030

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence 0 to 17 (Fold 1)

Dependence 18 to 29 (Fold 1)

Dependence 30 to 49 (Fold 1)

Dependence 50 to 69 (Fold 1)

Dependence 70 or Older (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 5_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: logloss
• num_class: 5
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

33.7 seconds

Metric details

	0 to 17	18 to 29	30 to 49	50 to 69	70 or Older	accuracy	macro avg	weighted avg	logloss
precision	1	0.547085	0.484404	0.434917	0.690106	0.593849	0.631302	0.610558	0.947229
recall	0.647416	0.266376	0.308051	0.573171	0.909762	0.593849	0.540955	0.593849	0.947229
f1-score	0.785978	0.358297	0.376605	0.494563	0.784855	0.593849	0.56006	0.580976	0.947229
support	658	458	857	1230	1219	0.593849	4422	4422	0.947229

Confusion matrix

	Predicted as 0 to 17	Predicted as 18 to 29	Predicted as 30 to 49	Predicted as 50 to 69	Predicted as 70 or Older
Labeled as 0 to 17	426	4	33	194	1
Labeled as 18 to 29	0	122	151	178	7
Labeled as 30 to 49	0	97	264	439	57
Labeled as 50 to 69	0	0	92	705	433
Labeled as 70 or Older	0	0	5	105	1109

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence 0 to 17 (Fold 1)

Dependence 18 to 29 (Fold 1)

Dependence 30 to 49 (Fold 1)

Dependence 50 to 69 (Fold 1)

Dependence 70 or Older (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• num_class: 5
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

18.2 seconds

Metric details

	0 to 17	18 to 29	30 to 49	50 to 69	70 or Older	accuracy	macro avg	weighted avg	logloss
precision	1	0.460177	0	0.38477	0.67068	0.557214	0.503125	0.488373	1.00623
recall	0.647416	0.340611	0	0.62439	0.913864	0.557214	0.505256	0.557214	1.00623
f1-score	0.785978	0.391468	0	0.476131	0.773611	0.557214	0.485438	0.503198	1.00623
support	658	458	857	1230	1219	0.557214	4422	4422	1.00623

Confusion matrix

	Predicted as 0 to 17	Predicted as 18 to 29	Predicted as 30 to 49	Predicted as 50 to 69	Predicted as 70 or Older
Labeled as 0 to 17	426	9	0	222	1
Labeled as 18 to 29	0	156	0	290	12
Labeled as 30 to 49	0	166	0	612	79
Labeled as 50 to 69	0	7	0	768	455
Labeled as 70 or Older	0	1	0	104	1114

Learning curves

Decision Tree

Tree #1

Rules

if (Birth Weight <= 250.0) and (Payment Typology 1 > 4.5) and (Payment Typology 1 <= 5.5) then class: 70 or Older (proba: 66.4%) | based on 4,958 samples

if (Birth Weight <= 250.0) and (Payment Typology 1 <= 4.5) and (APR MDC Description <= 22.5) then class: 50 to 69 (proba: 39.3%) | based on 3,814 samples

if (Birth Weight <= 250.0) and (Payment Typology 1 > 4.5) and (Payment Typology 1 > 5.5) then class: 50 to 69 (proba: 37.68%) | based on 2,033 samples

if (Birth Weight > 250.0) then class: 0 to 17 (proba: 100.0%) | based on 1,335 samples

if (Birth Weight <= 250.0) and (Payment Typology 1 <= 4.5) and (APR MDC Description > 22.5) then class: 18 to 29 (proba: 49.16%) | based on 1,125 samples

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence 0 to 17 (Fold 1)

Dependence 18 to 29 (Fold 1)

Dependence 30 to 49 (Fold 1)

Dependence 50 to 69 (Fold 1)

Dependence 70 or Older (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back